Bridging the Guideline Implementation Gap: A Systematic, Document-Centered Approach to Guideline Implementation

Design: This approach uses the Guideline Elements Model (GEM) to represent the guideline knowledge. Implementation requires a number of steps to translate the knowledge contained in guideline text into a computable format and to integrate the information into clinical workflow. The steps include: (1) selection of a guideline and specific recommendations for implementation, (2) markup of the guideline text, (3) atomization, (4) deabstraction and (5) disambiguation of recommendation concepts, (6) verification of rule set completeness, (7) addition of explanations, (8) building executable statements, (9) specification of origins of decision variables and insertions of recommended actions, (10) definition of action types and selection of associated beneficial services, (11) choice of interface components, and (12) creation of requirement specification. Results: The authors illustrate these component processes using examples drawn from recent experience translating recommendations from the National Heart, Lung, and Blood Institute’s guideline on management of chronic asthma into a workflow-integrated decision support system that operates within the Logician electronic health record system. Conclusion: Using the guideline document as a knowledge source promotes authentic translation of domain knowledge and reduces the overall complexity of the implementation task. From this framework, we believe that a better understanding of activities involved in guideline implementation will emerge. j J Am Med Inform Assoc. 2004;11:418–426. DOI 10.1197/jamia.M1444. Clinical practice guidelines provide a rich source of up-todate knowledge about best clinical practices. Guidelines can reduce inappropriate practice variation, speed the translation of research into practice, and support quality and safety initiatives. However, guideline knowledge must be implemented before it can be expected to influence clinicians’ behavior. Implementation is the phase in the guideline lifecycle in which strategies, systems, and tools are created to operationalize the knowledge and recommendations set forth by the guideline developers. In a computerized environment, implementation involves a number of steps to translate the knowledge contained in guideline text into a computable format and to integrate the information into a clinical workflow. Unfortunately, a gap exists between the information contained in published guidelines and the knowledge and information that are necessary to implement them. Those who are charged with operationalizing guidelines must bridge this implementation gap. Currently, implementers use poorly specified, largely tacit knowledge acquisition processes and a multiplicity of guideline knowledge representations to create computable decision support systems from published guidelines. This approach results in considerable inconsistency in the encoding of guideline knowledge and in the functionality of the systems that are created. In some cases, Patel et al. have found that different recommendations would be given for the same patient using computable representations of the same guidelines that are formulated by different individuals. This work describes our progress toward understanding the activities that are required to bridge the ‘‘guideline implementation gap.’’ We describe a generic process for translation of document-based knowledge into workflow-integrated decision support tools. Our goal is to define a set of replicable activities that will help to systematize the process and make its component activities explicit. We describe this approach as document-centric because the guideline document is used as the authentic source of domain knowledge. This approach differs from that applied by many current models, which design clinical decision support based on designers’ Affiliations of the authors: Yale Center for Medical Informatics, Yale University School of Medicine, New Haven, CT (RNS, GM, AE); Yale New Haven Hospital, New Haven, CT (ET). Supported by National Library of Medicine grants 1 R01-LM-07199, R29-LM05552, and T15-LM07065 Correspondence and reprints: Richard N. Shiffman, MD, MCIS, Yale Center for Medical Informatics, 300 George Street, Suite 501, New Haven, CT 06511; e-mail: . Received for publication: 08/22/03; accepted for publication: 05/03/04. 418 SHIFFMAN ET AL., Guideline Implementation Gap by gest on Jauary 8, 2016 ht://jam ia.oxfournals.org/ D ow nladed from understanding of the guideline authors’ goals or intentions and their conceptualization of guideline logic. Background Tierney et al described their frustration in creating a computer-based implementation for an evidence-based guideline to assist with management of heart failure. That guideline, like many others, lacked explicit definitions, focused on omission errors (rather than errors of commission), and did not account for comorbid conditions, concurrent drug therapy, or timing of interventions. Peleg et al. observed specialty society experts as they created flowcharts based on narrative guidelines and found excessive ambiguity and problems in sequencing. In addition to these problems, guidelines are often incomplete, i.e., they regularly fail to describe appropriate behavior for an exhaustive set of situations that may befall practitioners. Moreover, conditional recommendations are regularly undecidable, i.e., they fail to specify in a clear, consistent manner the parameters on which decisions are based. Likewise, actions may not be executable. Often, the level of abstraction at which decision variables and actions are described is inappropriate for implementation. Grol et al. found that clinicians were considerably less likely to adhere to vague and nonspecific recommendations. For optimal implementation, all guideline recommendations must be integrated with clinical workflow, a principle characterized by widespread variability of practice. Tu et al. have devised a ‘‘deployment-driven’’ approach to integration of guideline recommendations into workflow that is currently being tested. Cabana et al. have created a useful conceptual framework that describes critical barriers to successful implementation, including awareness of and familiarity and agreement with guideline content, and clinicians’ self-efficacy, outcome expectancy, and ability to overcome inertia of previous practice. Attention to knowledge deficits and attitudinal issues are also critical in the design of successful systems. Guideline Elements Model: A Reusable Guideline Document Model Weuse the Guideline ElementsModel (GEM) as a computable representation for clinical practice guidelines. GEM is an XML-based document model that uses a multilevel hierarchy to store the heterogeneous kinds of information contained in clinical practice guidelines, including identification of the guideline’s developer, description of the development process, definition of the guideline’s purpose, the intended audience, the target patient population, and the recommendations themselves. The hierarchy contains more than 100 tags by which guideline information can be classified (marked up) and modeled at varying levels of abstraction. GEM was conceived and built in XML and, therefore, can take advantage of each of the emerging XML-related technologies. GEM facilitates translation of guideline information and knowledge into a format that can be processed by computers while remaining readable by humans. American Society of Testing andMaterials (ASTM) International has adopted the GEM Document Type Definition as a standard representation for guidelines using XML. GEM has been used successfully to assist with guideline quality appraisal and to translate guideline recommendations into Arden syntax. Gershkovich and Shiffman demonstrated the feasibility of guideline implementation from a GEM file by automatic generation of Web forms for data entry, but this system was not integrated into clinical workflow. Georg et al. recently demonstrated the superiority of GEM encoding over traditional methods as a first step in the development of a knowledge base from textual documents. Formulation of the Guideline Implementation Model In the currentwork, we set a short-term goal of understanding whatmeta-information is commonly necessary to supplement guideline-derived knowledge for computer-based guideline implementation. The implementation model was formulated empirically during several experiences in the design and development of guideline-based decision support tools. As such, it reflects the longitudinal experience of one group operating on a variety of guidelines over more than a decade. In earlywork,wemarkedup anumber of guidelines to gain an overview of the kinds of information contained in them. Next, we envisioned how decision support systems designed to implement the guideline knowledge might operate. We then characterized the kinds of information that would be necessary to move from the guideline markup to a fully operational decision support system, i.e., to bridge the implementation gap. Several principles governed our design activities. d The approach should be systematic, replicable, and reusable. d Knowledge acquisition using markup is an iterative process, in which narrative text is refined and clarified. d The final implementation of the guideline knowledge must be closely integrated with workflow. In general, workflow is site specific. To encourage integration with workflow, we advocate offering end-users a decision support systemwith as many services as possible to offset the inevitable downside associated with using any information technology. Effective decision support designs integrate guideline knowledge with beneficial services that are appreciated by users, consider the volume of advice and prioritize it so as not to overwhelm users, and employ effective user interface design principles. Use of an electronic health record system or physician order entry system offers an opportunity to integrate patient-specific, guideline-prescribed advice into the clinician–patient interaction. d Any local adaptation of the guideline knowledge must be transparent, i.e., an audit trail must be constructed. d The information systems designers who build the decision support tools need not possess high-level informatics skills. d The process should result in output generalizable to multiple platforms. To optimize the generalizability of the model, we chose to not create tools that directly incorporate the processed guideline knowledge into the local information system. Instead, the output we are aiming for is a detailed requirements specification to serve as a starting point for an iterative process that can be applied in a variety of information environments. Model Description and Validation The proposed document-centric model describes the translation of guideline knowledge and the acquisition of relevant metainformation into a framework that can be operationalized 419 Journal of the American Medical Informatics Association Volume 11 Number 5 Sep / Oct 2004 by gest on Jauary 8, 2016 ht://jam ia.oxfournals.org/ D ow nladed from within a computer-mediated decision support system. Generically, the meta-information necessary for implementation falls into two main categories: (1) information that more precisely specifies the guideline knowledge and (2) information that facilitates workflow integration. In this section, we describe the model and, at the same time, provide details of our experience with implementation of an asthma guideline that validates the model. The model provides a design pattern that reduces the overall complexity of the translation task by specifying combinations and sequences of activities that are crucial to accomplish the task. We describe a set of generic activities that takes as input a narrative guideline and provides as output a detailed specification for a workflow-integrated decision support system. We draw examples from our recent experience in translating recommendations from the National Heart, Lung, and Blood Institute’s guideline on management of asthma into a workflow-integrated decision support system that operates within the Logician (GE Medical Systems, Hillsboro, OR) electronic health record system at the Yale New Haven Hospital. We gained considerable experience over the last several years, initially implementing parts of this guideline on handheld computers and ultimately deploying a decision support system throughout the ambulatory and inpatient pediatric units at Yale NewHaven Hospital. As currently deployed, the chronic asthmamanagement systemprompts for the collection of a patient’s symptom information, interprets these findings and suggests classification of asthma severity and level of control, facilitates prescription of appropriate medications and referrals when indicated, and provides a handout with customized management advice—all based on guideline content. An overview of this approach to guideline implementation is shown in Figure 1. The remainder of this section describes the component activities. In the indented text, we describe how our experience with the asthma guideline demonstrates that the model can be applied in the real world. Guideline text extracts are shown in italics. Guideline Selection To initiate the implementation process, a specific practice guideline must be chosen as a knowledge source. In many cases, several developers will have created guidelines on the same topic. The guideline selection process is based on user or organizational imperatives, e.g., identified areas in which there is exceptional local practice variation, areas in which new knowledge ought to be put into practice, or areas in which resource use is inappropriate. At this early juncture in the implementation process, the validity of the guideline as well as its likely implementability must be considered. Instruments from Shaneyfelt and the AGREE Collaboration can be used to assess guideline quality. Guidelines that have been reported according to the COGS statement are likely to be easier to assess for validity because they systematically report precise details that are critical for understanding a guideline’s development, its recommendation statements, and potential issues in its application. Some guidelines are likely to be operationalized more readily than others. We are developing an Implementability Rating Profile (IRP) that helps users consider several intrinsic guideline attributes that affect implementation success. The IRP instrument examines decidability, executability, effects on process of care, measurable outcomes, flexibility, and several other dimensions to assist users in anticipating barriers to implementation. Asthma has been recognized as a major problem in the community served by the Yale Primary Care Center (PCC), with more than 600 admissions for asthma occurring annually for children alone. There was a widespread perception among clinicians and administrators that asthma care could be improved. The National Heart, Lung, and Blood Institute (NHLBI) released an evidence-based guideline on asthma in 1996 that has achieved wide use and acceptance. A committee of PCC clinicians, administrators, and information systems personnel selected this guideline for implementation. When applying a document-centric approach to guideline translation, the individual recommendation, not the guideline as a whole, is the essential unit of implementation. Therefore, a secondary selection step is to choose specific recommendations within the selected guideline to be implemented. Most guidelines contain several recommendation statements, and some may be beyond the scope of a current effort. The implementation effort at the Yale Primary Care Center was directed at increasing the appropriate use of pharmacologic interventions for chronic asthma management, facilitating pulmonary specialty referrals, and enhancing patient education. Three specific guideline-prescribed tasks seemed amenable to automation: d Classification of asthma severity asmild-intermittent or as mild-, moderate-, or severe-persistent based on clinical findings d Choosing appropriate medications, based on classification, level of control, age, and current medications F i g u r e 1. An activity diagram that displays processes involved in guideline translation. 420 SHIFFMAN ET AL., Guideline Implementation Gap by gest on Jauary 8, 2016 ht://jam ia.oxfournals.org/ D ow nladed from d Providing a take-home message in the form of a customized handout. For this implementation, users elected to not operationalize those guideline components that dealt with diagnosis of asthma or management of acute exacerbations. Markup If the implementer plans to use XML tools to facilitate operationalization, the first step after guideline selection is markup. (This is the only step in the translation process that is specific to an XML-modeling approach.) Guideline knowledge components relevant to operationalization are identified and tagged. In addition to the Knowledge Components subhierarchy in GEM (which includes recommendations, definitions, and algorithm hierarchies); additional elements that describe the guideline’s purpose, intended audience, target population, and schemas for rating evidence quality and recommendation strength are usually valuable for implementation. Guidelines most often define recommendations as imperatives, i.e., activities applicable to the entire eligible population, or as conditionals, i.e., activities recommended in specifically defined circumstances. Conditionals can be expressed as IF 1⁄2decision variableðsÞ have valueðsÞ THEN 1⁄2actionðsÞ where decision variables and their values describe antecedent conditions that must be fulfilled if a recommendation is to be applicable, and actions describe consequents that are recommended under these circumstances. Imperatives, on the other hand, are stated simply as: